In this paper, a fully immersive serious game system that combines two Natural User Interfaces (NUIs) and a Head Mounted Display (HMD) to provide an interactive Virtual Environment (VE) for patient rehabilitation is proposed. Patients extquotesingle~data are acquired in real-time by the NUIs, while by the HMD the VE is shown to them, thus allowing the interaction. A Long Short-Term Memory Recurrent Neural Network (LSTM-RNN), previously trained by healthy subjects (i.e., baseline), processes patients extquotesingle~movements in real-time during the rehabilitation exercises to provide the degree of their performance. By comparing the functionalities of the proposed system with the ongoing state-of-the-art, it is worth noting that the reported fully immersive serious game system provides a concrete contribute to the current literature in terms of completeness and versatility. The results obtained by three rehabilitation exercises, chosen as reference case studies, performed on real patients affected by Parkinson extquotesingle s disease have shown the effectiveness of the presented approach. Finally, the analysis of the feedbacks received by the therapists and patients who have used the system have highlighted remarkable results in terms of motivation, acceptance, and usability.

VRheab. A fully immersive motor rehabilitation system based on recurrent neural network / Avola, Danilo; Cinque, Luigi; Foresti, Gian Luca; Marini, Marco Raoul; Pannone, Daniele. - In: MULTIMEDIA TOOLS AND APPLICATIONS. - ISSN 1380-7501. - (2018). [10.1007/s11042-018-5730-1]

VRheab. A fully immersive motor rehabilitation system based on recurrent neural network

Avola, Danilo
Primo
;
Cinque, Luigi;Marini, Marco Raoul;Pannone, Daniele
2018

Abstract

In this paper, a fully immersive serious game system that combines two Natural User Interfaces (NUIs) and a Head Mounted Display (HMD) to provide an interactive Virtual Environment (VE) for patient rehabilitation is proposed. Patients extquotesingle~data are acquired in real-time by the NUIs, while by the HMD the VE is shown to them, thus allowing the interaction. A Long Short-Term Memory Recurrent Neural Network (LSTM-RNN), previously trained by healthy subjects (i.e., baseline), processes patients extquotesingle~movements in real-time during the rehabilitation exercises to provide the degree of their performance. By comparing the functionalities of the proposed system with the ongoing state-of-the-art, it is worth noting that the reported fully immersive serious game system provides a concrete contribute to the current literature in terms of completeness and versatility. The results obtained by three rehabilitation exercises, chosen as reference case studies, performed on real patients affected by Parkinson extquotesingle s disease have shown the effectiveness of the presented approach. Finally, the analysis of the feedbacks received by the therapists and patients who have used the system have highlighted remarkable results in terms of motivation, acceptance, and usability.
2018
rehabilitation; virtual reality; human-computer interaction; sensors data fusion; deep learning
01 Pubblicazione su rivista::01a Articolo in rivista
VRheab. A fully immersive motor rehabilitation system based on recurrent neural network / Avola, Danilo; Cinque, Luigi; Foresti, Gian Luca; Marini, Marco Raoul; Pannone, Daniele. - In: MULTIMEDIA TOOLS AND APPLICATIONS. - ISSN 1380-7501. - (2018). [10.1007/s11042-018-5730-1]
File allegati a questo prodotto
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1085317
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 35
  • ???jsp.display-item.citation.isi??? 31
social impact